Skip to main content
Feedback

Converting an existing cloud cluster to elastic scaling

About this task

This topic covers the prerequisites, procedure, and configuration needed to convert an existing runtime cloud to use Elastic scaling in Kubernetes.

The high level steps are:

  1. Install and configure a Kubernetes cluster. (Skip this step if already running Kubernetes.)
  2. Update runtime properties to enable elastic executions.
  3. Stop all cloud cluster nodes.
  4. Create Kubernetes resources for the Boomi runtime including namespace.
  5. Install and configure the Boomi elastic controller service.
  6. Configure runtime helm chart and deploy to start cloud cluster.
note

This does not cover migrating from a different type of runtime (e.g. basic runtime or runtime cluster) to an elastic runtime cloud.

Prerequisites

Kubernetes cluster

You can use any Kubernetes cluster, as long as the you (or the authorized user) have permissions to create the following resources:

  • Namespace
  • Deployment
  • HorizontalPodAutoscaler
  • Role
  • RoleBinding
  • Service
  • ServiceAccount
  • StatefulSet

Managed Kubernetes solutions are available from various providers for these capabilities.

kubectl

kubectl is a command line tool which used for running commands against a Kubernetes cluster. Several steps in this document rely on this utility. Instructions for installing kubectl can be found in the Kubernetes documentation-installing kubectl.

Helm

Helm is a package manager for Kubernetes that lets us create, distribute, and deploy applications to a cluster via helm charts. Boomi provides two helm charts to set up elastic executions: runtime-cloud and runtime-elastic-controller. Both of these helm charts are hosted on Docker Hub.

To install Helm, refer to the Helm documentation.

KEDA

Kubernetes Event Driven Autoscaler (KEDA) is a tool that can scale the workers based on custom application metrics. Additionally, it lets us scale the workers to 0. Refer to the Scaling elastic cloud clusters topic for more details.

KEDA needs to be deployed to our cluster in order to scale the workers efficiently. To install via Helm, refer to the Keda documentation.

Required Backups

Prior to any changes, ensure the following files are backed up:

  • \<installation-directory\>/bin/atom
  • \<installation-directory\>/bin/atom.vmoptions
  • \<installation-directory\>/conf/container.properties

Conversion steps

The following are the steps you will need to follow to complete the migration

  1. Update properties
  2. Namespace
  3. Persistent Volume
  4. Persistent Volume Claim
  5. Elastic Controller
  6. Elastic Controller configuration
  7. Boomi Runtime cloud
  8. Runtime Cloud configuration

1. Update properties

In the platform UI, go to Manage > Runtime Management > select the cloud cluster > Properties panel > Advanced tab.

In this Advanced tab, update the following properties:

  • "Execute Processes as Forked JVMs" to "ELASTIC"
  • "Java Class Path Prepend" to /usr/lib/jvm/java-11-amazon-corretto/lib/tools.jar
    • The Java Class Path now points to the JDK embedded within the container image

When these properties are saved, click No, restart later when asked to restart the runtime. Instead, we will stop the runtime entirely.

If the runtime is in a VM-based setup, stop all running nodes one at a time to ensure graceful shutdown:

<installation-directory>/bin/atom stop

If the runtime is already in Kubernetes, scale down all the cloud pods:

kubectl scale sts <statefulset-name> -n <namespace> --replicas=0

The next steps will install the necessary components in the Kubernetes cluster.

2. Namespace

All of the Boomi related resources are created in the boomi namespace. Create this namespace by running the following command:

kubectl create namespace boomi
note

If you install multiple runtime clouds, each cloud should be in its own namespace. Use unique namespaces accordingly.

3. Persistent Volume

A Persistent Volume is needed for the Persistent Volume Claim which is used for the shared file system required for the runtime. You are responsible for creating the Persistent Volume, since it is heavily dependent on the storage technology.

For customers deploying in AWS, the following links provide more information for using EFS in EKS:

4. Persistent Volume Claim

A Persistent Volume Claim with the name of runtime-claim should be created in the corresponding namespace. This points to the shared file system of the existing runtime cloud.

If configured correctly, the created Persistent Volume Claim will have a status of Bound.

5. Elastic Controller

The elastic controller is a new Boomi runtime service responsible for creating the appropriate Kubernetes resources for forked executions and workers. The helm chart and the image for this application are hosted in Docker Hub.

  1. Create a separate file for the helm values of the elastic controller. For example, elastic-controller-helm-values.yaml. The contents of this file should be as follows:
fullnameOverride: runtime-elastic-controller
  1. Deploy the elastic controller via the helm chart:
helm upgrade --install --wait --namespace boomi --create-namespace -f elastic-controller-helm-values.yaml runtime-elastic-controller-release oci://docker.io/boomi/runtime-elastic-controller --version <version>
  1. Verify the state of the elastic controller pods in the boomi namespace:
kubectl get pods -n boomi -l app=runtime-elastic-controller

1. Elastic Controller Configuration

Because the elastic controller manages the Kubernetes resources required for the integration processes, it provides several hooks to customize how these resources are created.

Run the command below to list the helm values.

helm show values oci://docker.io/boomi/runtime-elastic-controller --version <version>

Pay special attention to these fields:

  • boomi.runner: This customizes the forked execution pods
  • boomi.worker: This customizes the worker pods.
  • boomi.browser: This customizes the browser pods
  • boomi.execution: This customizes the forked execution pods, worker pods, and the browser pods

You can add these fields to the elastic-controller-helm-values.yaml file; then run the helm upgrade command for these changes to be reflected.

Only new Kubernetes resources created by the elastic controller will be affected; existing resources will remain as they are.

6. Boomi Runtime Cloud

This example details how to start an existing runtime using the helm chart.

  1. Create the runtime-cloud-helm-values.yaml file for the helm values of the runtime cloud. The contents of this file are as follows:
fullnameOverride: runtime-cloud
boomi:
  atomName: <name-of-your-existing-runtime-cloud>
  containerPropertiesOverrides: 'com.boomi.container.processExecMode=ELASTIC'
  1. Deploy the runtime cloud via the helm chart:
helm upgrade --install --wait --namespace boomi --create-namespace -f runtime-cloud-helm-values.yaml runtime-cloud-release oci://docker.io/boomi/runtime-cloud --version <version>
  1. Verify the state of the runtime cloud pods in the boomi namespace:
kubectl get pods -n boomi -l app=runtime-cloud

If the cloud pods are not up and running, verify that your persistent volume claim contains the runtime installation directory in the correct location. Since the helm chart automatically mounts the specified persistent volume claim to /mnt/boomi, ensure your persistent volume claim is structured so that the installation directory is accessible at /mnt/boomi/<installation-directory>.

1. Runtime Cloud Configuration

Similar to the elastic controller’s helm chart, customizations are possible via the helm chart of the runtime cloud as well.

Helm values can be listed by running the following command:

helm show values oci://docker.io/boomi/runtime-cloud --version <version>

These fields can then be added to the elastic-cloud-helm-values.yaml file. Run the helm upgrade command again for these changes to reflect.

Conversion is now complete.

On this Page